Parallelism in Structured Newton Computations

A fundamental computational procedure in practically all areas of scientific computing is the calculation of the Newton step (in n-dimensions). In many cases this computation represents the dominant cost in the overall computing task. Typically the Newton step computation breaks down into two separable subtasks: calculation of the Jacobian (or Hessian) matrix along with the right-hand-side, and then the solution of a linear system (which, in turn, may involve a matrix factorization). Both subtasks can be expensive though in many problems it is the first, calculation of the function and derivative matrices, that dominates. In most cases when the Newton step computation is relatively expensive, the function that yields the Newton system is itself a ‘structured’ computation. A structured computation is one that breaks down into a (partially ordered) straight-line sequence of (accessible) macro computational subtasks. For example, if F is a function that is computed by evaluating the sequence F1, F2, F3, in order, then F is a structured computation. The general structured situation can be described as follows: F is a structured computation, z = F (x), if F is evaluated by computing a (partially-ordered) sequence of intermediate vectors y defined below: Solve for y1 : F 1 (x, y1) = 0 Solve for y2 : F 2 (x, y1, y2) = 0 .. .. Solve for yp : F p (x, y1, y2, · · · , yp) = 0 “Solve” for output z : z − F p+1(x, y1, y2, · · · , yp) = 0    . (1)